Temporal Relational Modeling with Self-Supervision for Action Segmentation

Temporal relational modeling in video is essential for human action understanding, such as recognition and segmentation. Although Graph Convolution Networks (GCNs) have shown promising advantages relation reasoning on many tasks, it still a challenge to apply graph convolution networks long sequences effectively. The main reason that large number of nodes (i.e., frames) makes GCNs hard capture model temporal relations videos. To tackle this problem, paper, we introduce an effective GCN module, Dilated Reasoning Module (DTGRM), designed dependencies between frames at various time spans. In particular, via constructing multi-level dilated graphs where the represent from different moments video. Moreover, enhance ability proposed model, auxiliary self-supervised task encourage module find correct wrong Our DTGRM outperforms state-of-the-art segmentation models three challenging datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), Breakfast dataset. code available https://github.com/redwang/DTGRM.